Expression snippet transformer for robust video-based facial expression recognition

Although Transformer can be powerful for modeling visual relations and describing complicated patterns, it could still perform unsatisfactorily video-based facial expression recognition, since the movements in a video too small to reflect meaningful spatial-temporal relations. To this end, we propose decompose of into series snippets, each which contains few frames, then boost Transformer’s ability intra-snippet inter-snippet modeling, respectively, obtaining Expression snippet (EST). For devise an attention-augmented feature extractor enhance encoding subtle snippet. introduce shuffled order prediction head corresponding loss improve motion changes across subsequent snippets. The EST obtains state-of-the-art performance, demonstrating its superiority other CNN-based methods. Our code trained model are available at https://github.com/DreamMr/EST